Collaborative Project Management with a Web-based Database Editor

Managing large projects in a cooperative fashion with geographically distributed partners is a complex process. Support tools are required for information sharing and they must permit synchronized, collaborative actions. Actions are triggered by timed events, by automated assessment and control modules, by traditional joint reviews of visualizations of the project state, or other means. They result in new planing steps, re-scheduling of milestones and resources, mailings of alerts, highlighting of critical sections in the project plan. Here, we describe an approach relying on an active, object-relational database which generates visualizations of the global state and may include multimedia data. The visualizations are distributed to the clients via applets for common Web browsers. Moreover, through socket connections, clients can also navigate and edit the global state using a cursor paradigm. Process control is provided with scripts, based on extensions of Tcl/Tk. The proposed language allows event management, establishes communication links and permits message passing, provides database access and comes with a comfortable visualization toolkit. 1 The Need for Collaborative Multimedia Tools In a global economy, large scale projects involve partners which are geographically distributed. To plan and monitor a project‘s progress, task dependencies must be identified, timing requirements be set down, resources limitations made public, etc. For a continuous review of the project‘s state, all partners require a consistent shared view of all data. This view may include multimedia data like still video shots of construction sites, pictures of defective parts, video and audio recordings of witnesses, etc. Any changes to these data must be done in a transactional fashion to maintain integrity of the overall planning. This is the classical task of a database management system (DBMS). Storing the project data in a consistent way and permitting transactional access to them, however, is only a small part of project management. Design review and refinement is to a large extend a social process. It involves negotiations, plan alternations, external reviews, feasibility studies, debates over priorities and strategies, what-if scenarios, changes at the last minute. These long and costly interactions must be supported by suitable tools for synchronous, distributed collaborative management which forms a subdomain of Computer Supported Collaborative Work (CSCW). Other domains which require collaborative multimedia tools are team science and tele-medicine as mentioned in the comprehensive overview by Prakash et al. [9]. Current technology tends to focus on either asynchronous groupware, i.e. email exchange and placement of data into intranets, or conferencing tools with video transmissions, shared whiteboards on which a person can draw, and chat rooms. Commercial products supporting these modes are Lotus Notes, Microsoft NetMeeting, and CoolTalk [9]. All CSCW tools adhere to a relaxed WYSIWIS model (What You See is What I See) to establish a common ground for discourse, but they differ in what can be shown, whether objects can be manipulated by each participant, how communication between users occurs and how much each participant is aware of each others location, activities, focus of attention, etc. It is generally agreed that the initial brain storming phase profits most from video conferences and shared whiteboards. They are important for establishing a social contact and for quick sketches, but they remain strangely detached from the actual objects of discourse. In contrast, we aim at a database managed, visualized, shared object space similar to what is called multiple shared data artifacts in [9] and implemented as TupleSpace in [7]. This approach makes use of the fact that project management, once it has passed the initial kick off, is based upon an abundance of highly structured objects: documents, maps, design drawings, dependency graphs, time and resource scheduling tables, budget plans, etc. Views of this space (views in the database as well as in the interface sense) need to be distributed and planning tools must provide synchronized access to the project data. In the next Section we outline our approach and emphasizes the difference between picture passing and operation passing. Next we present Gantt diagrams as an example of a derived view. The solution relies on TclDB, which is a navigational language which ties DBMS, visualization and distribution over the Web together. Finally, we discuss the architecture of the client-server solution, review our proposal and point out further research areas. Users are invited to connect to http:// www.db.informatik.uni-kassel.de/~escher/tcldbTclet/Welcome.de.html to get a first impression even though some of the concurrency features are not operational yet. 2 Shared Object Spaces View sharing for the WYSIWIS paradigm, also called GUI sharing in [9], includes the use of a telepointer and can be achieved by some picture passing mechanism similar to the X-window display protocol. This is sometimes called the centralized approach because the actual objects remain at the central collaboration server. Examples for this form are NetMeeting, XTV, SharedX, and Jupiter. Alternatively, one can choose a replicated approach which caches local copies of the collaboration items. This requires elaborate synchronization protocols and is similar to distributed transaction systems [11]. Most CSCW text editors work according to this principle. Examples for the replicated mode are DistEdit, Grove, GroupKit and MMConf, all referenced in [9]. A third approach, which we adhere to, is to work with a centralized object server which distributes visualization operations to be performed at the collaboration clients and which in turn accepts navigational and editing commands from the participants. We call this operation passing to contrast it to view passing. We believe that, much like in display generation through command passing (e.g. in PostScript), it can save bandwidth and can easily be adapted to user preferences. In our prototype, operations are an encoding of a suitable, generic interaction paradigm for navigation and object editing. Based on our experience with a shared database editor, called ESCHER (12, 13, 15, 16, 17), we use a cursor paradigm which seems familiar to anybody who has ever been in contact with a text processor. Cursors, which we call fingers to distinguish them from the mouse cursor, highlight objects of interest within the shared object space. Navigation allows entering complex objects (in operation), escaping to the enclosing object (out), going to the next or previous object within a collection (next, back), etc. Operations on objects include insert, delete, cut & paste, query, etc. All operations require one or more fingers which belong to participating (human) users or autonomous agents. Putting the collaborative project management tool into operation should not require elaborate installation procedures. Bearing in mind that we aim at geographically distributed partners with heterogeneous associations, — from in-house employees to an external sub-contractor making a bid in the acquisition phase — the Web forms the ideal base for wide-spread use, an opinion which we share with [7]. Unfortunately, the Web mode of information supply is stateless, i.e. it follows the connect-get-close paradigm of information interchange and provides mostly pre-compiled pages where contents are possibly fetched from a DBMS [6]. Thus the Web is not directly geared towards synchronous collaboration. As Fielding et al. put it with regard to HTTP and the Web: „Although this simple model of communication scales well for simple retrieval tasks, it is not sufficient for the complex interactions in software engineering (or in any collaborative work process). [1, p. 87]“ Luckily, through the use of applet technologies, i.e. small mobile code loaded from a server and executed at the client side in a so-called plug-in, we can establish a connection with a server database through which updates can flow in a bi-directional way. In our case we base our development on John Ousterhout‘s Tcl/Tk [8], where applets are called Tclets1, but other solutions based on commonly available plug-ins are equally possible. The following steps describe the remaining architectural issues. • Collaboration data (e.g. project data) are placed into a DBMS; currently we use our database editor ESCHER as a server (to be possibly replaced by a proper object-relational DBMS at a later stage); the use of complex objects, as offered by the object-relational model, seems important since it tends to give a more intuitive view of relations. • The server provides scripts written in an extension of Tcl, called TclDB, which collect and update data in the DBMS. The scripts also generate various application specific displays. • This display code, which is pure Tcl/Tk, is shipped in small chunks to clients as needed. There the code is executed in a safe interpreter. The shipped code includes call-back routines which are scripts containing TclDB code. This code cannot be executed at a client and would not be understood by the Tcl plug-in of the browser. • Events occurring at the client, e.g. mouse clicks, key strokes, etc. cause these call-back commands to be shipped back to the server via socket connections. The included TclDB commands are understood and executed by the server. • A client may start operations by clicking directly into tables or into any derived visualizations, say a diagram. Events relate to the active finger of a user. The finger itself is positioned on an atomic or complex object. If the event occurs in a derived visualization, the finger there has a counterpart within the (nested) source DBMS table and the movement is reflected in this table. • Concurrency is given through multiple fingers. The area where a finger operates is called a nimbus. The area which a user 1. pronounce ”Ticklets“ currently pays attention to is called the focus which forms a viewport over the data space. • Unlike in a traditional DBMS, users are not isolated from concurrency but are made aware of it. Concurrency awareness requires that actions of other users (their fingers), which stretch into a user‘s focus, are signalled. The terms focus and nimbus and their possible interrelations were introduced in [10]. 3 Interactive GANTT Diagrams To demonstrate the construction principle, we pick time management as an example within project planning, although this example has little multimedia content. Note that we assume familiarity with these well-known project management techniques and we like to stress that none of the basic algorithms, say for critical path computation, and none of the basic diagramming techniques are new and need to be elaborated here. Rather we aim at showing how these well-known techniques can be integrated into a collaborative, database driven tool. As is standard practice, projects are broken up into sets of tasks (jobs, subprojects, project steps, work packages). Each task is assigned an identifier, a description and a duration in some suitable unit of time, mostly working weeks. Other relevant data, like resources and responsibilities, estimated risk factors, etc. are also common, but are usually stored in other tables related to the time schedule through foreign keys (here TASKID) or direct links. Tasks are related to each other. A particular task A (and possibly others) must be finished before task B (and possibly others) can start. This can be represented by drawing an arc from A to B. In a nested table we may either include a set of successor tasks within the tuple for task A or alternatively a set of predecessor tasks for task B. In the table (Fig. 1) we follow the second, more common approach because managers tend to know what they require for their subprojects but are less knowledgable about later steps of others. The object-relational view of the table includes additional, computed fields, namely earliest starting time (ES), earliest finishing time (EF), latest starting time without risking project delay (LS), latest finishing time without risking project delay (LF), and ISOTIME, the day of year computed from LF for milestones and deliverables. Figure 1: View of a shared interactive project table through a Tcl-enabled WWW browser ES, EF, LS, and LF are incremental data. Also, by convention, we define duration zero for milestones. Two of these are predeclared, namely START and END for project kick off and project close down. Our tool inserts these into the table should they not be present and it connects all tasks without predecessors (empty REQUIRES-set) to START, respectively connects all tasks which do not appear in any REQUIRES-set of another task (see Fig. 1) with END. Once the dependencies are established and are checked for absence of cycles (the dependency graph must be a directed acyclic graph), we create a list of tasks and sort this list topographically1. Tcl is of great help here because it offers associative arrays, i.e. we can use TASKID as reference to a field. An upward and a downward sweep through the list follows to compute earliest start and finish times, respectively latest finish and start times. Tasks with identical earliest and latest start, respectively finishing times are said to be on the critical path. Otherwise the task is said to have a non-zero slack. A suitable visualization for the resulting timing of the project is a Gantt Diagram. Fig. 2 shows our implementation of this well-known technique. Here, tasks are shown as horizontal bars. If they are part of a critical path, they have a red rim around the bar. If not, their slack time is added in a lighter shaded color. Milestones are denoted by a diamond and have the ISOTIMEvalue attached to the left. All this is fairly standard and is offered by most other commercially available packages. However, this diagram is reactive and shared by users for synchronous collaboration! Any user can establish a personal finger which may point to a particular task. This task is then shown as a colored bar, respectively simultaneously as a colored tuple in the table (see the darker tuple with TASKID = “FDESIGN“ in Fig. 1). Among all existing fingers, exactly one finger is active at any one time. The data related to the active task are shown in the left part of the diagram (Fig. 2). The data shown there can be edited by the owner of the active finger who is said to have floor control. Floor control can be passed on to other users on request in a kind of token-passing. When data are changed, say a new duration must be entered because of a task delay or new estimate, the data are passed to the table and are updated there. Then re-computation is triggered and the resulting Gantt diagram is redrawn. From our current experience, there is a small, but noticeable delay in the update of the diagram for projects with more than a dozen global tasks. Computations could be sped up by representing arcs directly by links (database pointers, possible through ESCHER‘s type 1. The topological order is a partial order and needs not be unique. If the nodes are placed down in this semi-order, all arcs go to the right. When seen in a Gantt diagram (see Fig. 2) a task depends only on tasks shown as bars above it. Figure 2: Gantt diagram generated from project table above link) rather than by foreign keys as shown here. Fingers are placed onto tasks by clicking with the mouse either on the bar in the diagram or by dragging the mouse with the left mouse button pressed over parts of the tuple in the table (within our interaction settings, an individual click places the finger always on an atomic field, dragging the mouse places it on the smallest enclosing semantic object, i.e. a tuple, a set or list of atomic values or tuples, the entire table). Dragging the mouse with the left button pressed over a bar in the Gantt diagram stretches the bar and sends the new computed duration to the table. When the button is released, new critical path computations are triggered and the diagram is re-drawn as mentioned before. This interaction mode is extremely useful in collaborative time planning. Engineers can point out dangerous passages. Others can comment on what they see and can themselves point to resulting effects, say a deliverable moving into the holiday season or stretching past a contract date. When tasks are adjusted as part of a risk analysis, the resulting table can be stored as materialized views, e.g. as best, worst and most-likely case scenarios. Other diagrams, like PERT diagrams and histograms can be drawn in a similar fashion using the scripting technique discussed in further detail below. Because the development is based on interpreted Tcl, rapid prototyping is possible and new applications are added easily. Also, extensions to project budget control, quality management, resources management, etc. are obvious. Visualization in monitoring is another useful mode. Applications of this type are e.g. in automated process control, where database tables are keeping track of semi-autonomous robots which periodically update their positional data and download new orders [18]. Using the options menu, users can also adjust their display, e.g. the scaling of time units, the diagram spacing, etc. Also, zooming and unzooming allows the user to enter into the task details which are recursively structured with start and end subtasks. Again, the nested table approach comes in handy with a recursive, essentially infinite schema allowing subtask definitions to arbitrary levels of detail. 4 TclDB: A Navigational Database Language The glue which makes database, visualization and Web distribution stick together is TclDB, which stands for Database Tool Command Language [12, 17] and is not to be confused with Tcl-DP (Tcl Distributed Processing, http://www.sco.com/ Technology/tcl/Tcl.html). As mentioned before, TclDB is an extension of Ousterhout‘s Tcl/Tk development [8]. In essence, TclDB is a scripting language for running through nested tables, collecting and updating atomic and complex values. Its operational mode is a mixture of navigational and declarative statements, the latter being issued through path expressions which may contain predicates. Some commands are shown in Table 1 which should be self explaining. A full list of the 30 odd commands and other examples can be found in [12]. A small sample of actual code, here from collecting all required predecessor tasks for a given task, is shown below. # Returns a list of all predecessors # of a task. pf is on tuple which Table 1 Examples of TclDB-commands Commands (partial) Remark fid push -first -last -where predicate path with an existing finger fid move on first (default) enclosed object, resp. move finger on last enclosed object move finger on object satisfying predicate, returns 0/1 move finger along a path including attributes and indexes fid pop move finger fid to enclosing object fid get [path] return value on which finger rests, optionally extended by path starting with finger position fid set value [path] -tonull [path] -toempty [path] set value for atomic object on which finger fid sits set complex object to database null turn a set-valued object into an empty set # describes task. Assumes dag consistency has been checked. proc prj_requires {pf} { ;# parameter pf is finger set requires {} ;# initialize to empty list $pf push REQUIRES ;# go with pf to REQUIRES attribute if [$pf isnull] {error “NULL requires Set“} ;# shouldn‘t be undefined if ![$pf isempty] { ;# ok if empty nothing to collect $pf push -first ;# go to first element lappend requires [$pf get] ;# append foreign key to list while [$pf go -next] { ;# iterate over set until past last lappend requires [$pf get] } $pf pop ;# go from last element to set REQUIRES } $pf pop ;# go from set REQUIRES to tuple return $requires ;# return collected list } Further details of the prototype code, which we put into the public domain under the GNU licensing agreement, can be inspected at our WWW site. 5 The Web Client Side Solution When run through the Web, a client in our collaborative project management tool is a Tclet for a WWW Browser with a Tcl plug-in. A common browser which can be Tcl-enabled is Netscape Version 4.0 or higher. In case such a plug-in is needed, it can be loaded from e.g. http://sunscript.sun.com/products/plugin.html. This client creates a reactive interface, namely the project table and/or diagrams like the Gantt diagram shown here. This interface is created as a normal Tcl/Tk widget tree and may use most of the features of Tcl/Tk, except for the following: • access the client file system (except possibly /temp) • use libraries (package require) • use system calls (exec) • grab and hold the screen focus • connect via sockets to other servers except the one from which the page with the Tclet came. These restrictions stem from safety requirements for interpreters running on client machines and apply to other plug-ins, say for Java applets, as well. They actually do not restrict our approach with the possible exception of writing to safe storage as part of a distributed transaction concept, e.g. 2-Phase Commit. Widgets have command scripts bound to them which are executed when events occur for these widget. In general, the scripts contain TclDB commands which the client neither understands nor would be allowed to execute since they might access table data which resides on disk. Thus the concept is not that of a distributed database nor does the server do any page shipping and there is no local database functionality at the client. 6 The Web Server Side Solution The server directs the client by shipping Tcl-Code as needed. The connection to the client requires thus no extra protocol: we use TclDB for client to server dialog and Tcl/Tk for server to client communication. Also, the client is quickly started since it only needs an initial Tclet of about 3 KB with a basic eval loop. When up and running, response is quick as messages (operation passing, not picture passing!) tend to be small. As an example, a push command with a finger id and an object identifier for the current position would require less than 50 bytes and would trigger a major re-display with the focus moving onto the first task of a project. Fig. 3 shows the timing of the client-server connection and some of the Tcl commands used in establishing the communication. The socket identifiers used by server and client are sid and cid, respectively. The method configure triggers a redisplay once the received command have gone through the eval-loop at the client. What we cannot show here for lack of space are a number of details. One concerns e.g. object identification. ESCHER, like most object-relational DBMS, uses internally some form of unique object identifiers. Tcl, on the client side, relies on hierarchical names to identify widgets and also operates with associative arrays. Through a simple protocol agreement, the client can be instructed to generate names which are an encoding of ESCHER object identifiers shipped by the server. As the need arises, they are transmitted back to the server which can decode them to find out which objects are involved in an operation. Programming the connection itself was actually straightforward because Tcl’s socket command is a marvel of simplicity as compared to the raw UNIX system programming interface. Also Tcl/Tk’s asynchronous nature with file events governing the arrival of data, well described in [4], blends much better into the communication concept as developments in C or C++. 7 Conclusion and Open Issues We have argued for collaborative multimedia tools which can be applied to project management. We have proposed a Webbased solution which operates on nested tables for project data and can generate derived views through attached scripts. A working prototype based on our database editor ESCHER and the navigational language TclDB exists and produces e.g. reactive visualizations like Ganttand PERT-diagrams as shown in Fig. 2. Some aspects, such as concurrency awareness, are not fully operational yet. One reason is that ESCHER has no extended transaction concept yet which would be suitable for CSCW applications. Unlike in ACID-transactions, isolation must give way to concurrency awareness. With multiple fingers we can simulate this, but this includes no synchronization. To offer full concurrency, the server should be multi-threaded. However, Tcl itself is (was) not multi-thread safe. Modifications of Tcl, like Tcl-DP Distributed Processing, Pttcl to support P(osix)Threads in Tcl(7.6), or Mtcl which gives access to the multi-threading features of Solaris 2 and which includes a modified version of Tcl 7.4b3, can handle multiple threads. Some of these specialized developments should be made obsolete by the upcoming releases of Tcl which supports multiple threads. We are currently looking into its use. There are alternative mobile code languages, in particular Java, which is in wide-spread use and is also multi-thread safe. Furthermore, Oracle8i can store compiled Java code in tables and ship it from there much like we can store TclDB scripts in our nested tables. However, there is the added complexity of having TclDB for the server and Java for the client. In addition, we end up with large applets which cannot be extended on a per need base. All these considerations make us reluctant to change. Some of the disadvantages might disappear through the use of JavaScript [2] and closer integration of Java and Tcl in the form of Jacl and Tcl Blend [5] or SpecTcl and SpecJava [14]. Other languages which could be shipped to a client are O’Caml, Limbo, Obliq, and Telescript. Because they either don’t have the powerful GUI features of Tcl/Tk or the market share of Java, we feel they are not an alternative for Web-based CSCW. Another interesting research issue touches upon safety questions. As mentioned before, the Tclet runs in a safe interpreter. This is ensured automatically through the browser. What is returned by the client depends on what the server has issued before as command scripts which are bound to widgets. Thus they seem to be safe. However, the client code runs in user space and can be manipulated. A malignant user could substitute some of these scripts with dangerous TclDB code and trigger its execution on the server. Thus, the server must have ways of securely identifying the Server Client start TclDB-Server socket -server \ load Web page, ship Tclet (about 3 KB) Web page